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Abstract 


This  final  report  summarizes  briefly  work  done  to  formulate  and  test  some 
cognitively-based  principles  useful  for  designing  instruction  in  scientific  or 
engineering  fields.  These  principles  stress  particularly  the  coherence  and 
interpretability  of  the  knowledge  acquired  by  students.  The  principles  were 
specifically  applied  to  analyze  the  underlying  knowledge  and  thought 
processes  needed  to  interpret  scientific  concepts  effectively;  to  investigate  how 
actual  experts  and  novice  students  interpret  such  concepts;  and  to  devise  and 
test  instruction  designed  to  teach  such  concepts  more  effectively.  Attempts 
were  also  made  to  devise  computational  environments  to  explore  and 
implement  these  instructional  ideas. 


Introduction 


The  goal  of  this  project  was  to  formulate  and  test  some  cognitively-based 
principles  for  designing  instruction  in  complex  domains,  such  as  science  or 
engineering. 

These  instructional  principles  were  to  be  based  upon  the  following:  (a)  An 
analysis  of  the  underlying  knowledge  and  thought  processes  needed  for  good 
performance  in  the  particular  domain  of  interest,  (b)  An  adequate 
understanding  of  the  knowledge  and  thought  processes  of  students  coming  to 
the  learning  situation.  The  instructional  process  would  then  be  explicitly 
designed  to  transform  a  student's  knowledge  and  thinking  so  as  to  approximate 
those  needed  for  good  performance. 

To  implement  the  project,  it  was  planned  to  focus  on  a  subject-matter 
domain  which  would  be  sufficiently  limited  in  scope  to  be  investigated  in  some 
detail,  but  which  would  be  representative  of  the  complexities  encountered  in 
broader  realms  of  science  or  engineering.  Accordingly,  we  chose  to  devote 
primary  attention  to  the  interpretation  and  learning  of  scientific  or  mathematical 
concepts  (such  as  the  physics  concept  "acceleration").  Indeed,  an  adequate 
understanding  of  such  concepts  is  of  fundamental  importance  to  any  scientific 
work  and  is  an  essential  prerequisite  for  any  problem  solving  in  scientific 
domains.  The  learning  of  such  concepts  presents  many  difficulties  for  students 
and  conventional  teaching  methods  are  often  quite  ineffective. 

In  carrying  out  our  work,  we  were  centrally  interested  in  obtaining 
information  sufficiently  detailed  to  elucidate  underlying  thought  processes  and 
to  assess  the  ways  in  which  they  would  be  affected  by  specific  instructional 
interventions.  Accordingly,  we  concentrated  our  efforts  on  detailed 
observations  of  individual  subjects  under  controlled  conditions. 


General  Theoretical  Considerations 

Analysis  Qt  desirable  parformanca 

Our  basic  criterion  for  ultimately  desired  intellectual  performance  is  that  a 
person's  knowledge  should  be  effectively  usable  so  that  it  can  be  applied 
correctly  and  flexibly  in  a  wide  range  of  situations. 

This  performance  criterion  is  far  from  trivial.  Indeed,  all  too  commonly  the 
knowledge  acquired  by  students  as  a  result  of  ordinary  instruction  is  merely 
"nominal";  i.e. ,  it  can  be  displayed  by  students  on  standard  examination 
questions,  but  cannot  be  used  flexibly  by  them  to  enhance  their  capabilities  of 
independent  thinking. 

The  criterion  of  effective  usability  raises  the  following  theoretical  questions: 
What  are  some  of  the  essential  characteristics  needed  to  make  knowledge 
about  some  topic  effectively  usable?  Our  analysis  suggested  that  the  following 
characteristics  are  of  central  importance: 


(1)  The  topical  knowledge  must  be  accompanied  by  ancillary  knowledge 
ensuring  that  the  topical  knowledge  can  be  properly  interpreted.  Such 
interpretation  knowledge  must  specify  how  to  instantiate  the  topical  knowledge 
in  any  particular  case,  and  how  to  determine  whether  such  instantiation  is 
correct  or  not.  (Such  interpretation  is  clearly  an  essential  prerequisite  for  more 
complex  problem  solving.) 

(2)  The  form  of  the  knowledge,  i.e.,  its  description  and  organization,  is  at 
least  as  important  as  its  content  to  ensure  that  the  knowledge  is  effectively 
usable.  In  particular,  it  is  essential  that  the  knowledge  is  adequately  coherent, 
i.e.,  that  knowledge  elements  are  interconnected  by  possible  (strict  or  plausible) 
inferences  among  them.  Such  coherence  greatly  facilitates  ease  of 
remembering,  ease  of  regenerating  knowledge  that  has  been  forgotten,  ease  of 
ensuring  consistency,  ease  of  debugging,  and  ease  of  extending  the 
knowledge  by  generalization  or  new  learning. 

(3)  Available  knowledge  needs  to  be  accompanied  by  adequate  control 
knowledge,  (a)  Such  control  needs  to  ensure  proper  invocation, 
implementation,  and  checking  when  available  knowledge  is  applied.  (Indeed, 
particularly  in  science,  disciplined  knowledge  application  is  a  prerequisite  for 
accurate  and  flexible  knowledge  use.)  (b)  Deliberate  and  automatic  control  of 
knowledge  must  be  well-integrated.  Indeed,  performance  of  a  complex  task 
would  be  very  difficult  unless  some  lower-level  knowledge  can  be  used  nearly 
automatically  in  ways  that  are  fast  and  effortless  (so  as  to  leave  adequate 
mental  capacity  available.for  the  performance  of  more  complex  aspects  of  the 
task). 


InstrugtiQnalJmpiications 

The  preceding  considerations  lead  to  the  following  general  instructional 
implications: 

(1)  Instruction  should  be  designed  so  as  to  explicitly  ensure  that  any 
knowledge  acquired  by  students  have  the  preceding  characteristics  needed  for 
good  performance. 

(2)  Learning  can  be  facilitated  by  maximizing  these  characteristics  at  any 
intermediate  stage  of  the  learning  process.  (For  example,  care  devoted  to 
keeping  any  newly  acquired  knowledge  interpretable  and  coherent  can 
appreciably  facilitate  extension  of  this  knowledge  by  further  learning.) 

The  following  paragraphs  summarize  briefly  how  we  tried  to  implement  and 
investigate  these  general  theoretical  guidelines  in  the  specific  domain  of 
scientific  concepts 


Analysis  of  Concept  Interpretation 

An  essential  task  in  dealing  with  any  scientific  or  mathematical  concept 
(such  as  "acceleration",  "area",  ...)  involves  the  ability  to  interpret  this  concept 
so  as  to  instantiate  it  properly  in  any  specific  case.  This  interpretation  task  is 
actually  more  complex  than  one  might  naively  suspect. 

To  interpret  such  a  concept  one  must  retrieve  from  memory  some  knowledge 
stored  about  the  concept,  and  then  engage  in  information  processing  to  apply 
this  knowledge  in  the  particular  situation  of  interest.  Different  possible  modes  of 
concept  interpretation  are  then  possible,  depending  on  the  kind  of  knowledge 
stored  in  memory  and  the  kind  of  processing  done  with  it.  Each  of  these 
possible  modes  has  some  distinctive  characteristics,  in  terms  of  unambiguity, 
precision,  generality,  and  ease  of  use.  Correspondingly,  each  of  them  can  also 
lead  to  distinctive  kinds  of  errors. 

We  performed  an  analysis,  buttressed  by  some  observations,  of  various 
possible  modes  of  concept  interpretation  and  their  implications.  In  brief,  these 
modes  of  interpretation  can  be  classified  into  the  following  two  types: 

(1)  Some  interpretation  modes  are  "formal”,  i.e.,  deliberately  designed  to 
achieve  great  precision  and  generality.  They  store  in  memory  well-specified 
and  general  definitional  knowledge  about  the  concept,  knowledge  which  must 
then  be  explicitly  processed  to  be  applied  in  any  specific  instance.  The 
knowledge  stored  may  be  a  "declarative  specification"  of  features  characterizing 
the  concept;  the  processing  needed  for  application  in  particular  instances  may 
then  require  appreciable  problem  solving.  Alternatively,  it  may  be  a  "procedural 
specification"  which  provides  a  more  detailed  and  explicit  specification  of  how 
to  interpret  the  concept  in  any  particular  instance.  Such  formal  interpretation 
modes  have  the  advantage  of  being  reliably  accurate,  precise,  and  general. 
They  have  the  disadvantage  that  their  application  is  slow  and  requires 
significant  mental  effort. 

(2)  Other  interpretation  modes  are  based  on  a  repertoire  of  case-based 
knowledge..  The  knowledge  stored  in  memory  consists  then  of  knowledge 
about  various  special  cases,  standard  cases,  and  typical  kinds  of  cases  of  the 
concept.  If  this  knowledge  has  become  compiled  as  a  result  of  familiarity  and 
repeated  use,  such  knowledge  can  then  be  invoked  almost  automatically  by 
recognition  processes  that  match  this  knowledge  with  any  particular  instance  of 
interest.  These  non-formal  case-based  interpretation  modes  have  the 
advantage  of  efficiency,  i.e.,  they  can  be  applied  quickly  and  effortlessly.  They 
have  the  disadvantage  that  they  can  easily  lead  to  errors  or  inconsistencies, 
and  are  not  well  suited  for  making  general  inferences. 

What  then  is  an  "ideal"  mode  of  concept  interpretation  which  is  both  reliably 
effective  and  efficient?  The  preceding  analysis  suggests  that  such  an  ideal 
mode  of  concept  interpretation  should  rely  on  both  formal  and  case-based 
knowledge  used  in  complementary  ways.  If  one  encounters  a  familiar  situation, 
it  is  most  efficient  to  use  case-based  knowledge,  and  then  to  rely  on  more 


formal  knowledge  to  check  the  resulting  interpretation  if  necessary.  On  the 
other  hand,  if  one  encounters  an  unfamiliar  situation,  tries  to  detect  or  diagnose 
errors  or  inconsistencies,  or  needs  to  make  general  arguments,  then  it  is  most 
useful  to  rely  on  formal  knowledge  and  to  use  case-based  knowledge  for 
possible  check-points. 

An  account  of  the  preceding  work  was  published  as  an  ONR  report  (Reif, 
1986).  An  article  based  on  this  work  has  also  been  accepted  for  publication  in 
the  journal  Cognitive  Science  (in  press). 


Comparative  Studies  of  Good  and  Poor  Performance 

To  obtain  data  bearing  on  the  characteristics  of  usable  knowledge,  and 
more  specifically  on  the  preceding  analysis  of  effective  and  efficient  concept 
interpretation,  we  made  some  detailed  observations  to  elucidate  the  nature  of 
the  underlying  knowledge  and  thought  processes  used  by  persons  who  were 
either  good  or  poor  interpreters  of  scientific  concepts. 

To  this  end,  we  (predominantly  Lisa  Quinn  and  I)  constructed  two  carefully 
designed  sets  of  questions  requiring  persons  to  interpret  the  concept 
"acceleration"  qualitatively  in  a  variety  of  situations.  The  persons  used  in  this 
experimental  investigation  were  either  physics  professors  who  had  recently 
taught  an  introductory  mechanics  course  dealing  with  the  concept 
"acceleration",  or  students  enrolled  in  such  a  course.  Each  such  person  was 
individually  asked  to  talk  out  loud  about  his  or  her  thinking  while  trying  to 
answer  the  questions  —  once  without  interventions  by  the  experimenter,  and  a 
second  time  with  requests  for  fuller  explanations  of  his  or  her  thinking.  The 
transcript  of  the  person's  tape-recorded  utterances,  together  with  his  or  her 
written  work,  constituted  then  a  protocol  which  was  subsequently  analyzed  in 
detail. 


Those  experts  (i.e.,  physics  professors)  who  are  good  at  interpreting  the 
scientific  concept  seem  to  behave  in  a  manner  similar  to  that  proposed  by  our 
ideal  model  of  concept  interpretation,  (a)  They  can  interpret  reliably  the 
knowledge  invoked  by  them,  (b)  Their  knowledge  is  quite  coherent,  i.e., 
consistent  and  usable  for  various  inferences,  (c)  They  have  both  formal  and 
case-based  knowledge,  and  they  use  both  these  kinds  of  knowledge  in 
complementary  fashion. 

However,  even  these  good  experts  (who  answer  correctly  more  than  90%  of 
all  questions  given  to  them)  occasionally  make  mistakes.  Many  of  these  are 
due  to  the  experts’  inadequate  control  of  their  available  knowledge  (e.g.  to 
failure  to  invoke  appropriate  knowledge,  even  when  it  was  recently  invoked  in  a 
slightly  different  context,  or  failure  to  check  adequately  application  of  this 
knowledge). 


Even  when  students  are  currently  enrolled  in  a  course  where  they  have 
studied  and  repeatedly  applied  the  concept  "acceleration"  for  more  than  a 
couple  of  months,  their  ability  to  interpret  this  concept  is  quite  poor.  In  our 
investigation,  these  students  were  unable  to  answer  correctly  about  60%  of  the 
qualitative  questions  about  this  concept. 

Analysis  of  the  protocols  reveals  that  students'  underlying  knowledge  about 
this  concept  has  the  following  major  characteristics: 

(1)  Their  knowledge  is  quite  incoherent,  consisting  largely  of  various 
disconnected  knowledge  fragments  (many  of  them  incorrect)..  As  a  result, 
students  are  often  unable  to  deal  with  situations  deviating  slightly  from  standard 
situations  previously  encountered  by  them;  they  fail  to  detect  inconsistencies; 
and  they  encounter  paradoxes  which  they  cannot  resolve. 

(2)  Correct  concept  interpretation,  relying  on  on  many  such  fragmentary 
knowledge  elements,  would  require  that  each  such  knowledge  element  be 
accompanied  by  specific  applicability  conditions.  However,  not  surprisingly, 
students  often  either  fail  to  store  adequately  such  applicability  conditions,  or  fail 
to  retrieve  them  when  invoking  a  knowledge  element. 

(3)  Students  rarely  invoke  general  formal  knowledge  and  are  often  unable  to 
articulate  such  knowledge.  Furthermore,  when  they  do  invoke  such  knowledge, 
they  are  frequently  unable  to  interpret  it  and  thus  cannot  apply  it  successfully  in 
particular  instances. 

In  short,  students'  knowledge  seems  to  lack  most  of  the  characteristics  of 
interpretability  and  coherence  which  we  have  identified  as  essential  to  the 
effective  usability  of  knowledge. 


Nominal  versus  flood  experts 

There  are  "nominal"  experts  (so  designated  by  criteria  such  as  title,  position, 
degrees,  or  other  credentials)  whose  actual  performance  is  not  particularly 
good.  Hence  it  is  important  to  specify  whether  nominal  criteria  or  actual 
performance  data  are  used  to  select  "experts"  in  any  comparative  study  of 
experts  or  novices.  For  example,  in  our  experiments  we  found  "experts"  (i.e. , 
physics  faculty  members  at  a  university")  who  were  not  good  at  interpreting  an 
elementary  concept,  like  "acceleration",  which  they  themselves  had  recently 
taught..  Indeed,  their  performance  was  partly  reminiscent  of  that  of  novices. 
Such  observations  have  not  only  sociological,  but  also  cognitive  interest.  In 
particular,  one  can  try  to  trace  the  performance  deficiencies  of  such 
experienced  persons  to  specific  deficiencies  in  the  form  of  their  underlying 
knowledge. 


Some  results  of  these  comparative  studies  of  good  and  poor  performance 
have  been  summarized  in  a  recently  published  paper  (Reif,  1987).  We  hope  to 
present  a  more  extensive  account  in  a  future  publication. 


Instructional  Studies 

The  preceding  theoretical  ideas  and  observations  about  the  essential 
characteristics  of  flexibly  usable  knowledge  suggest  some  specific  guidelines 
for  the  design  of  instruction. 

Working  with  myself  and  Lisa  Quinn,  Peter  Labudde  (a  postdoctoral  visitor 
from  Switzerland)  carried  out  an  instructional  experiment  to  test  some  of  these 
ideas.  To  be  specific,  we  wanted  to  investigate  the  instructional  efficacy  of 
implementing  merely  the  following  two  design  guidelines  based  on  our 
cognitive  considerations:  (a)  Ensure  explicitly  the  coherence  and 
interpretability  of  newly  acquired  knowledge,  (b)  Ensure  that  this  new 
knowledge  is  also  coherent  with  students'  preexisting  knowledge,  and  can  be 
used  to  debug  this  knowledge. 

To  implement  these  guidelines  for  the  teaching  of  a  new  concept 
("acceleration”),  the  instructional  intervention  did  the  following:  (a)  It  taught 
students  an  explicit  procedural  specification  of  the  concept  acceleration  and 
then  gave  students  some  practice  in  applying  this  procedure  in  several  diverse 
situations.  (Specification  of  the  concept  in  terms  of  a  procedure  was  designed  to 
make  the  new  knowledge  interpretable.  The  consistent  application  of  this 
single  procedure  to  all  cases  was  intended  to  make  the  new  knowledge  highly 
coherent  )  (b)  It  then  asked  students  to  use  this  procedural  specification  to 
detect,  diagnose,  and  correct  errors  previously  committed  by  themselves  or 
errors  purportedly  committed  by  others.  (By  having  to  use  their  newly  acquired 
coherent  knowledge  to  confront  their  own  prior  notions  or  other  common 
misconceptions,  students  were  supposed  to  restructure  their  entire  knowledge 
in  coherent  form.  They  were  also  thereby  learning  quality-control  needed  to 
detect  and  correct  deficiencies  in  their  new  and  preexisting  knowledge.) 

The  instructional  intervention  in  this  experiment  lasted  about  half  an  hour 
and  resulted  in  the  following  main  outcomes:  (a)  Students’  abilities  to  interpret 
the  concept  acceleration  improved  markedly,  from  about  40%  correct 
interpretations  in  a  pretest  before  the  instruction,  to  about  95%  correct  in  a 
posttest  after  the  instruction,  (b)  In  their  attempts  to  interpret  the  concept 
acceleration,  students  invoked  many  incorrect  knowledge  elements  before  the 
instruction,  but  invoked  essentially  no  incorrect  knowledge  elements  after  the 
instruction,  (c)  Students’  interpretation  of  the  concept  after  the  instruction  was 
both  effective  and  efficient.  In  simple  situations,  they  relied  on  compiled 
knowledge  about  special  cases  of  the  acceleration;  but  in  more  complex 
situations,  they  reverted  to  the  procedural  specification  to  interpret  the  concept, 
or  to  check  tentative  answers  based  on  compiled  knowledge. 
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In  both  its  explicitness  and  emphasis  on  the  form  of  the  knowledge  acquired 
by  students,  this  instructional  intervention  differs  significantly  from  the  common 
methods  used  to  teach  scientific  concepts.  The  results  of  this  instructional 
experiment  were  also  certainly  encouraging,  although  not  definitive.  In 
particular,  this  instructional  experiment  incorporated  only  a  few  central  design 
principles  identified  by  our  analysis;  additional  tests  of  instructional  efficacy 
would  also  be  desirable. 

An  account  of  the  preceding  work  on  instruction  was  published  as  an  ONR 
report  (Labudde,  Reif,  &  Quinn,  1987).  An  article  based  on  this  work  has  also 
been  accepted  for  publication  in  the  European  Journal  of  Science  Education  (in 
press). 


Computer-implemented  Instruction 

Advantages  of  using  computers 

There  are  many  reasons  why  the  use  of  computers  would  be  advantageous 
in  formulating,  testing,  and  implementing  instructional  design  principles  of  the 
kind  discussed  in  the  preceding  paragraphs.  The  reasons  include  the 
following: 

Research  on  instruction  can  be  made  more  rigorous  by  translating 
instructional  designs  into  the  form  of  instructional  computer  programs,  (a)  Such 
programs  serve  to  make  more  explicit  the  assumptions  incorporated  in  an 
instructional  model,  (b)  They  allow  the  testing  of  such  models  under  controlled 
conditions  where  the  successive  actions  of  the  student  and  the  computer  tutor 
are  well-specified  and  well-known. 

The  computer  has  some  unique  advantages  as  a  medium,  (a)  It  can  provide 
powerful  graphical  representations,  including  dynamic  representations  of 
physical  phenomena  and  of  the  student’s  own  thought  processes.  (For 
example,  a  dynamic  graphical  representation  may  be  used  to  portray  a 
student's  own  progress  in  carrying  out  a  procedure.)  (b)  The  computer  can 
easily  store  and  then  redisplay  a  student's  past  work  on  an  intellectual  task. 
Thus  it  allows  the  student  to  reflect  on  his  or  her  past  thinking  —  and  also  more 
easily  to  detect,  diagnose,  and  correct  his  or  her  previous  deficiencies,  (c)  With 
careful  design,  computers  can  provide  an  environment  where  students  can 
actively  explore  new  concepts  while  being  constrained  to  behave  in  disciplined 
ways. 

Finally,  computers  would  be  very  valuable  for  delivering  practical  instruction. 
If  students  are  to  acquire  scientific  knowledge  that  is  actually  usable,  it  is 
imperative  that  they  become  actively  engaged  in  the  process  of  constructing 
their  own  knowledge.  The  computer  can  help  by  providing  exploratory 
environments  and  by  playing  the  role  of  private  individual  tutor.  By  contrast, 
most  usual  teaching  environments  fail  partly  because  they  put  students  in 
passive  roles  where  they  spend  most  of  their  time  merely  listening  or  reading. 
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Difficulties  of  computer  implementation 

The  previous  appealing  advantages  of  using  computers  are,  however,  not 
readily  realizable  because  of  several  major  difficulties  —  all  of  which  became 
very  evident  in  our  work  during  this  project. 

Some  of  these  difficulties  involve  computer  hardware.  The  implementation 
of  instructional  designs,  of  the  kind  discussed  in  the  previous  paragraphs, 
requires  computers  which  sufficiently  powerful  to  handle  text,  graphics,  and 
good  student  response  judging.  At  the  time  when  we  first  tried  to  use  computers 
for  this  project,  Apple  Macintosh  computers  just  became  available  and  seemed 
barely  adequate  to  satisfy  our  requirements.  Only  much  later  did  we  gain 
access  to  a  few  much  more  powerful  Xerox-1 108  workstations.  But  the  software 
available  on  relatively  new  machines  is  limited.  Furthermore,  one  quickly 
discovers,  to  one's  dismay,  that  even  expensive  and  powerful  machines  have 
basic  software  that  is  often  remarkably  "buggy" —  thus  making  any  work  very 
frustrating  and  progress  slow. 

Even  if  powerful  machines  were  readily  available,  with  well-functioning 
basic  software,  there  would  still  be  the  need  for  a  good  "authoring  environment" 
facilitating  the  design  and  implementation  of  instruction.  In  particular,  an 
instructional  designer  and  producer  must  be  able  to  direct  his  primary  attention 
to  fundamental  pedagogical  issues,  without  constantly  needing  to  deal  with 
details  of  low-level  programming.  The  designer  also  needs  to  be  able  to  easily 
edit  and  easily  modify  whatever  he  or  she  produces,  preferably  in  an 
environment  which  is  the  same  as  that  ultimately  seen  by  the  student  (i.e.,  which 
is  WYSIWYG,  "what  you  see  is  what  you  get"). 

During  the  project  we  did  not  have  available  anything  even  remotely 
resembling  such  an  authoring  environment  —  and  even  now  some  such 
environments  are  only  beginning  to  emerge.  Hence  we  spent  considerable 
time  and  programming  effort  trying  to  bring  into  existence  something  which 
might  begin  to  serve  our  needs. 


"Draw-Ed"  authoring  environment 

David  Oster,  one  of  the  programmers  working  on  the  project,  responded  to 
our  needs  and  suggestions  by  constructing  an  authoring  environment  exploiting 
the  Macintosh  computer. 

This  environment  uses  available  Macintosh  software,  "MacDraw",  as  an 
editor  which  can  construct  and  modify  various  kinds  of  objects,  including  text, 
graphics,  and  active  elements  (e.g.,  "buttons"  which  can  be  clicked  by  the 
mouse  to  produce  various  specified  responses).  In  this  way  one  can  readily 
construct  and  edit  "frames"  (i.e.,  complex  displays  ultimately  presented  to  a 
student)  in  a  way  where  the  author  can  see  exactly  what  the  student  will  see. 


This  editor  is  then  coupled  to  a  data-management  program  (implmented  by 
Oster  and  called  "DrawEd")  which  consists  of  the  following  components  to  carry 
out  various  functions: 

Information-transfer  between  editor  and  DrawEd  Using  "Switcher"  (another 
standard  Macintosh  software),  frames  created  in  the  editor  can  be  readily 
transferred  into  DrawEd,  or  transferred  back  to  the  editor  for  easy  modification. 

Data  base.  DrawEd  stores  all  frames  in  convenient  pictorial  form.  These 
picture  frames  can  be  easily  accessed  either  by  name,  or  by  browsing  through  a 
picture  file  where  any  picture  can  be  displayed  (either  full-size,  half-size,  or 
quarter-size)  and  then  visually  selected  with  the  mouse. 

Interpreter  of  active  elements.  Active  elements  in  the  MacDraw  editor  are 
accompanied  by  some  special  symbols  and  instructions.  DrawEd  interprets 
these  symbols  and  instructions  so  that  the  active  elements  actually  have  the 
specified  effects  when  selected  (e.g.,  so  that  buttons  selected  by  the  mouse 
actually  lead  to  the  display  of  specified  other  frames  or  windows). 

Tutor.  DrawEd  has  an  authoring  mode  and  a  tutorial  mode,  and  it  is  very 
easy  to  switch  between  these  two.  In  the  authoring  mode,  DrawEd  acts  like  the 
data-manager  just  described.  In  the  tutorial  mode  it  acts  as  a  tutor  interacting 
with  the  student  in  the  fashion  specified  by  the  instructional  designer.  Thus 
DrawEd  can  be  used  by  the  designer  for  authoring  and  modifying  instructional 
programs;  but  it  can  then  also  be  used  by  a  student  as  a  tutor  providing 
instruction. 

Transferto  LISP  machines.  Finally,  DrawEd  can  translate  all  the  information 
contained  in  the  frames  (i.e.,  text,  graphics,  and  instructions)  into  LISP  and 
transfer  this  information  electronically  (via  RS-232  interface)  to  a  Xerox-1108 
LISP  workstation.  This  transfer  capability  has  the  following  advantages:  If 
desired,  the  Macintosh  can  be  exploited  as  a  cheap  and  readily  available 
computer  acting  as  a  kind  of  "scratch-pad"  (or  electronic  story-board)  where 
instructional  designs  can  be  laid  out  and  perfected  up  to  a  certain  point.  Once 
this  is  done,  the  information  may  then  be  transferred  electronically  to  the  much 
more  expensive  LISP  machine  for  further  editing  and  refinements,  so  as  to 
exploit  the  much  greater  capabilities  of  those  machines. 


Final  status  of  computer  implementation 

The  Draw-Ed  authoring  system  became  barely  functional  only  near  the  end 
of  our  ONR  project.  Except  for  some  small-scale  testing,  the  project  was  thus 
not  able  to  exploit  this  system  for  significant  instructional  research. 


Continuing  Work 

Although  the  ONR  project  terminated  last  February,  some  of  the  lines  of  work 
initiated  under  this  project  are  continuing  under  different  auspices  (no  longer 
supported  by  the  ONR).  In  particular,  our  current  efforts  focus  more  broadly  on 
several  cognitive  issues  involved  in  the  learning  and  teaching  scientific 
concepts.  This  domain  of  investigation  seems  important  not  only  in  its  own 
right,  but  is  also  a  good  prototype  domain  within  which  to  explore  instructional 
principles  of  much  wider  applicability.  Since  Xerox  LISP  workstations  have 
become  more  available  to  us,  we  are  also  increasingly  trying  to  exploit  these 
machines  for  our  instructional  work  and  to  create  suitable  instructional 
authoring  environments  for  these  machines. 
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